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Abstract 

The  educational  literature  has  recorded  many  individual  advant-es 
in  technique  over  the  past  few  years.   Self-study  courses  have  been  dev- 
eloped.  Programmed  instruction  has  been  used.   Interactive  computer  pro- 
grams have  been  developed.   Professors  have  learned  to  change  dry  lecture 
techniques  to  more  stimulating  class  discussion.   However,  in  most  cases, 
these  advances  have  been  applied  one  at  a_  t  ime . 

The  course  design  discussed  here  sprang  from  the  understanding  that 
the  diverse  learning  objectives  which  underlie  a  well  rounded  and  well 
structured  course  lead  to  the  utilization  of  many  classes  of  learning  mate- 
rial.  These  in  turn  demand  diverse  learning  methods  to  be  utilized  -  with 
each  type  of  material  being  matched  by  its  most  effective  learning  method 
and  "delivered"  by  its  most  efficient  resource  -  professor,  student  or 
technology.   The  implementation  of  this  concept  is  reported  in  this  paper. 

The  result  was  an  integrated  use  of  the  available  learning  methods 
and  resources  -  synergistically  utilizing  the  best  of  each  resource  in  the 
learning  process.   Measurable  increases  in  student  performance  on  examina- 
tions, course  quality  (as  expressed  by  students),  and  academic  productivity 
all  resulted. 
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It  js  no  secret  that  the  cost  of  education  has  risen  much  faster  in 
recent  years  than  the  cost  of  living  as  a  whole.   At  the  same  time,  quality 
appears  to  have  fallen  off  -  or  at  least  perceived  quality  has  been  reduced. 
Students  complain  about  the  size  of  classes,  the  dedication  of  the  faculty  , 
to  research  rather  than  teaching,  the  dullness  of  some  classes,  and  many  other 

things.         y'    yf''^ 

All  of  these  symptoms  (despite  the  recent  dropoff  in  student  strikes  and 
other  dramatics)  may  be  indicative  of  what  some  have  termed  an  "educational 
crisis".   At  the  very  least,  they  are  symptoms  of  a  productivity  problem  in 
a  labor-intensive  industry.   l-Jhile  technology  has  done  much  to  raise  output 
per  man  in  many  other  industries,  it  has  done  little,  if  anything,  in  educa- 
tion.  A  lack  of  increased  productivity  not  only  causes  increased  costs,  it 
also  eliminates  the  possibility  of  increasing  quality  at  equivalent  cost. 
Under  these  conditions,  resources  are  most  often  strained  to  merely  "keep 
the  system  going." 

It  is  clear  that  the  productivity  problem  in  education  and  other  knowl- 
edge-based industries  now  is  of  extreme  importance.   As  Drucker  has  pointed 
out,  "The  bulk  of  tomorrow's  employnnent  will  be  in  service  trades,  knowledge 
jobs  -  in  health  care,  teaching,  government  ....  and  the  like.   And  no  one 
knows  much  about  knowledge  work,  let  alone  how  to  improve  it."  [1] 

Drucker  may  overstate  the  case  somewhat.   This  perceived  need  has  certain- 
ly called  forth  a  spate  of  research.   In  the  education  area  alone  (with  which 
we  are  concerned  in  this  paper),  there  has  been  a  significant  amount  of  research 
into  new  teaching  methods.   These  have  ranged  from  the  development  of  paper  pro- 
grammed instruction  textbooks,*  to  computer  assisted  instruction  [3,4],  to  com- 
puter-assisted homework  [5],  to  computer  managed  instruction,  to  on-line  model- 
ling of  significant  decision  processes  [6]. 


*See,  for  example,  [2]. 
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Other  efforts  have  turned  their  back  on  the  utilization  of  new  tech- 
nology and  instead  focused  on  the  behavioral  aspects  of  the  learning  pro- 
cess.  Chief  among  these  has  been  self-paced  self-study  learning,  one  of 
whose  chief  and  earliest  advocates  was  Keller.   Working  from  Skinnerian  prin- 
ciples, Keller  in  1965  ran  a  successful  application  of  self-study  [7].   Since 
then,  self-study  has  grown  and  flourished  although  in  universities  almost  en- 
tirely on  the  undergraduate  level.* 

In  addition  to  these  two  major  trends  -  computer  assistance  and  self- 
study  -  a  host  of  other  efforts  have  taken  place.   These  include  the  use  of 
video  tape,  pass-fail  courses,  smaller  classes,  larger  classes,  student  teach- 
ing and  so  forth. 

All  of  the  above  are  efforts  to  satisfy  a  felt  set  of  needs  which  might 
be  enumerated  as  follows: 

1.  To  increase  productivity  in  the  educational  process, 

2.  To  achieve  "higher  quality"  of  learning  (although  the  definition 
of  quality  is  unclear) , 

3.  To  "take  the  heat  off  the  student"  by  allowing  him  to  learn  mate- 
rial at  his  own  pace, 

4.  To  individualize  instruction  to  the  needs  of  the  particular 
student,  and 

5.  To  provide  the  student  with  more  autonomy  by  allowing  him  to  study 
in  depth  those  subjects  which  are  of  great  interest  to  him  while 
achieving  an  unpenalized  minimum  standard  only  in  other  courses. 

One  major  failing  of  these  efforts  has  been  the  lack  of  good  controlled 
studies  to  determine  what  has  been  accomplished.   In  the  main,  most  courses 
involving  new  techniques  such  as  self-study  have  been  composed  of  student  vol- 
unteers willing  to  take   the  educational  risks  of  experimental  situations.   In 


*See,  for  example  [8,9] 
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addition,  many  of  the  new  courses  are  based  on  techniques  which  explicitly 
do  not  allow  comparisons  with  control  groups.   Yet  there  is  an  evident  need 
to  evaluate  new  forms  of  education. 

Perhaps  most  interestingly,  most  educational  innovations  have  focused 
on  the  use  of  a  single  technique  at  a  time  as  a  particular  instructor  has 
become  enamorred  with  CAl,  self-paced  study,  or  some  other  individual  method 
[8,9].   This  has  been  perhaps  the  major  failing  of  the  process  of  innovation 
in  education.   Despite  much  jaw-boning  to  the  contrary,  very  seldom  has  a 
very  detailed  look  been  taken  at  the  existing  courses  in  today's  curriculums 
(especially  at  the  graduate  level).   If  this  were  done,  the  rudimentary  fact 
would  be  noted  that  most  courses  are  composed,  and  rightly  so,  of  diverse 
types  of  material  which  meet  multiple  learning  objectives.   It  follows,  then, 
that  diverse  learning  methods  should  be  applied  in  each  course  to  enable 
students  to  best  comprehend  the  diverse  material.   This  paper  reports  on  an 
experiment  in  the  integration  of  available  methods  "delivered"  by  varying  re- 
sources to  meet  the  requirements  of  diverse  learning  objectives  and  the  result- 
ing diverse  materials  in  a  single  course. 

The  objectives  -  material  -  methods  -  resources  development  process  of  an 
experimental  course  is  illustrated  in  the  next  major  section  of  this  paper. 
Much  has  been  written  about  the  need  for  clear  course  objectives,  tlie  decision 
process  with  regard  to  course  material,  and  the  pros  and  cons  of  different 
teaching  (or  learning)  techniques.   I  have  tried  to  avoid  repeating  all  of  this 
methodology  here.   Rather,  these  factors  are  presented  in  only  as  much  detail  as 
necessary  to  illustrate  the  development  of  the  particular  course  in  issue,  which 
in  turn  is  done  primarily  as  an  illustration  of  the  major  point  of  this  paper. 
That  point  is  simply  that  it  is  necessary  to  define  carefully  the  role  of  each 
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maior  resource  (professor,  student,  and  technology)  so  that  it  is  compatible 
with  the  prior  choices  of  course  objectives,  type  of  material,  and  learning 
methods.   The  choice  of  the  most  effective  and  efficient  resource  to  be  util- 
ized in  each  part  of  the  learning  process,  it  is  suggested,  can  lead  to  both 
greater  productivity  and  increased  quality  in  education.   The  second  major 
section  of  the  paper  presents  the  resulting  course  design  and  the  final  section 
reviews  the  results  obtained  with  regard  to  this  course  design  in  a  controlled 
experiment . 

I.   A  Framework  For  Course  Development 
Objectives  of  the  Course 

The  course  with  which  we  experimented  was  a  course  entitled  "Information 
and  Decision  Systems  I"  taught  to  both  undergraduate  and  graduate  students  at 
the  Sloan  School  of  Management  at  MIT.   The  subject  is  a  core  course  in  both 
curriculums.   It  is  intended  to  provide  students  with  an  in-depth  grounding 
in  the  principles  and  application  of  accounting  for  management  decision-mak.ng 
purposes.   It  introduces  financial  and  cost  accounting  systems  as  one  major 
segment  of  corporate  information  systems. 

The  course  has  several  objectives.   It  is  felt  that  the  student  should 
learn: 

1.  The  principles  which  underly  the  development  and  use  of  accounting  data. 

2.  The  process  of  management  decision-making  in  which  accounting  data  is 
used . 

3.  The  concepts  underlying  "management  planning  and  control"  -  the  other 
area  of  major  use  of  accounting  data. 

4.  The  detailed  knowledge  necessary  to  actually  perform  journalizing, 
adjusting  and  closing  entries ,  to  the  corporate  books. 

5.  The  process  of  cost  distribution,  product  pricing,  performance  analysis, 
and  capital  budgetting. 
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6.  The  basic  concepts  concerning  data  collection  and  the  "fit"  of 
the  accounting  system  with  a  model  corporate  information  system, 
and 

7.  The  subjectivity  and  imprecision  inherent  in  much  of  the  cost  and 
financial  data  used  by  management. 

Pedagogic  groupings  of  course  material 

With  regard  to  the  learning  process,  the  course  objectives  were  oper- 
ationally translated  into  teaching  material  which  divided  fairly  clearly  into 
the  following  six  Pedagogic  subsets: 

(a)  Material  illustrating  the  management  process  -  especially  the  de- 
cision making  process.   The  target  course  is  the  intial  "functional"  course 

in  management  for  most  of  the  Master's  candidates.   (Most  of  the  other  courses 
taken  in  the  first  teirm  are  disciplinary  courses.)   Some  idea  of  the  management 
process  must  be  depicted.   Emphasis  is  placed  on  the  primary  role  of  decision 
making. 

(b)  Motivating  material.  Accounting  methodologies,  no  matter  how  im- 
portant, can  be  dull.  There  is  a  need  to  insert  material  which  illustrates 
the  absolute  need  for  managers  to  understand  cost  and  other  financial  data. 

(c)  Material  concerned  with  the  "language" , "principles"  and  "procedures" 

of  accounting.   Making  up  the  bulk  of  the  course,  this  material  is  well  described, 
although  in  highly  varying  ways,  by  diverse  authors.*  Although  far  from  "cut 
and  dried,"  this  material  is  clearly  set  forth  in  understandable  form.   It  varies 
in  content  or  form  only  slightly  from  year  to  year. 

(d)  Material  on  which  the  student  can  test  his  detailed  knowledge  and 
understanding  of  accounting  techniques.   This  is  "practice"  material,  best 
popularly  exemplified  by  "homework." 

(e)  Material  highlighting  "grey  areas"  or  "major  issues"  which  face  the 

developers  and  users  of  financial  information.   The  language  and  procedures  of 

*Since  we  wished  to  illustrate  varying  views,  as  well  as  taking  the  best 
from  each  author,  we  utilized  [12,  13  and  14]  in  different  parts  of  the  course. 
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accounting  leave  a  great  deal  of  latitude  to  the  individual  corapany  accountant 
in  the  development  of  financial  results  for  any  period  and  in  the  development 
of  cost  data.   While  it  is  important  that  the  student  understand  the  basic 
postulates  and  principles  underlying  accounting  and  the  results  they  produce, 
it  is  perhaps  more  important  that  he  understand  the  "failings"  of  these  man- 
made  principles.   It  is  important  for  a  deeper  knowledge  of  the  material  for 
the  student  to  ponder  such  notions  as  replacement-cost  depreciation,  the  re- 
sults in  product  costing  of  various  methods  of  overhead  distribution,  and  the 
effect  on  the  balance  sheet  of  current  accounting  rules  in  times  of  inflation. 

(f )  Material  which  is  "timely",  has  changed  during  the  period  of  the 
course,  or  which  offers  new  perspectives  on  the  process.   In  general,  this  is 
material  which  cannot  be  (or  has  not  been)  put  into  writing  prior  to  the  start 
of  the  course.   It  is  material,  however,  which  often  adds  greatly  to  student 
understanding  of  the  course,  or  which  makes  the  course  material  "live"  through 
relating  it  to  current  events  or  to  ideas  which  the  professor  has  recently  dev- 
eloped and  is  excited  about. 

Available  learning  methods 

The  material  above  is  diverse.   It  therefore  demands  diverse  treatments. 
At  the  very  least  it  demands  a  careful  matching  of  the  way  that  each  type  of 
material  can  best  be  assimilated  against  the  repertoire  of  teaching  techniques 
available.   If  the  learning  process  were  better  understood,  instead  of  being  a 
coin-flipping  process  for  the  theory  which  one  wishes  to  select,  an  unambiguous 
raaterial-to-technique  matching  process  could  take  place.   Unfortunately,  it  is 
not  that  simple.   Today  each  professor  must  employ  his  own  heuristics  to  select 
techniques  for  each  type  of  material  to  be  learned. 
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A  learning  method  may  be  defined  as  a  procedure,  process,  or  technique 
by  which  material  is  presented  to  a  student  with  the  expectation  that  the 
student  will  gain  in  the  knowledge,  understanding  or  skill  encompassed  by  the 
material.   Many  learning  methods  are  available.   Use  was  made  in  the  new 
course  of  eleven  of  the  most  ubiquitous  methods.   These  eleven  learning  methods  - 
and  some  of  their  cardinal  attributes  from  our  point  of  view  -  are  as  follows: 

(1)   Listening  to  lectures:   excellent  for  communicating  new  or  ill- 
defined  material.   Very  poor  for  the  transmission  of  well-understood  facts. 

(2), (3)   Case  discussion  or  class  discussion  of  readings:   excellent  for 
the  involvement  which  they  provide,  the  requirement  which  they  place  on  the 
student  to  "think  through"  the  material  -  no  matter  how  cloudy  the  issue  - 
and  the  need  this  method  poses  for  the  student  to  determine  his  own  positim, 
and  to  defend  it.   Good  for  the  absorption  of  all  sides  of  major  issues  and 
for  the  exposition  of  all  sides  of  "grey  areas".   Poor  for  learning  "facts.'' 

(4)  Reading  of  textbooks:   Excellent  for  the  transmission  of  facts  and 
well-understood,  well-digested  material. 

(5)  Reading  of  programmed  instruction  (paper) :   A  first-rate  alternative 
for  some  students  to  traditional  textbooks  for  the  transmission  of  facts. 

(6)  Following  instructional  modules:   As  developed  for  self-study  courses, 
modules  are  "instructor's  guides"  to  sets  of  materials.   When  well-written,  they 
provide  directions  as  to  how  to  go  about  the  efficient  assimilation  of  material 
provided  by  others  of  the  above  learning  methods. 

(7),  (8)   Performing  homework  and  taking  marked  (but  ungraded)  quizzes: 
These  can  allow  the  student  to  test  his  knowledge  of  each  segment  of  the  work 
and  get  feedback  on  his  degree  of  knowledge. 


(9)  Taking  part  in  "question  sessions":  (These  are  class  sessions  in 
which  the  professor  or  a  teaching  assistant  may  he  present  but  which  are  de- 
signed to  be  "run"  by  the  students  who  introduce  the  topics  for  discussion. 
Only  those  topics,  which  meet  the  needs  of  the  students  at  the  time,  are 
discussed.   The  agenda  is  the  student's  .  .  .  not  the  professor's.)  Excellent 
process  for  original  learning  or  reinforcement  of  knowledge  for  some  students 
(see  later  discussion) . 

(10)  Computer-assisted  learning:   Very  poor,  to  date,  as  a  replacement 
for  textbooks  or  paper  programmed  instruction  for  original  learning  of  most 
material  on  a  cost-benefit  basis.   Highly  worthwhile,  if  the  programs  are 
well-developed,  in  allowing  students  to  test  their  understanding  of  material 
or  explore  the  effects  of  certain  choices  in  a  simulated  world  represented  by 
a  computer  model. 

(11)  Self-paced,  self-study:   A  "meta-method"  -  incorporating  many  of 
the  above  techniques  -  which  is  excellent  for  the  transmission  of  facts  and 
techniques  which  are  well  understood  and  where  the  material  or  adequate  quide- 
lines  to  its  comprehension  can  be  adequately  presented  in  written  form.   Self- 
study  suffers,  however,  in  areas  where  many  views  must  be  heard  to  illustrate 
the  multiple  facets  of  an  important  issue.   Self-pacing  may  also  be  mistaken  in 

a  professional  school  where  students  must  learn  to  "meet  deadlines"  as  train- 
ing for  later  life. 

As  previously  noted,  there  are  additional  learning  methods  not  discussed 
just  above.   And  we  have  only  begun  to  scratch  the  surface  in  developing  the 
multiple  attributes  of  some  of  these  methods.   The  prime  point,  however,  is 
that  diverse  learning  methods  do  have  diverse  strengths  and  weaknesses.   These 
attributes  of  each  method  must  be  matched  against  the  attributes  of  each  type  of 
material  to  be  presented  within  each  particular  course.   To  expect  that  one 
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learning  technique  can  be  stretched  to  cover  all  types  of  material  is  to 
deny  students  the  benefits  of  the  application  of  the  correct  tool  at  the 
correct  time.   It  is  similar  to  a  dentist  who  uses  a  single  cutting  bit,  or 
a  golfer  playing  with  only  his  five  iron.   The  task  can  be  accomplished,  but 
with  far  less  than  optimal  results  -  both  in  terms  of  output  and  aesthetics. 
From  methods  to  resources 

Interestingly,  the  eleven  teaching  methods  just  presented  can  be  broken 
down  into  three  major  classes  along  the  lines  of  the  three  major  resources 
available  today  to  the  learning  process.   The  methods  fall  into  classes  of 
those  that  are  professor  dominated,  those  that  are  heavily  dependent  on  the 
student  for  attainment  in  the  learning  process,  and  those  in  which  technology 
plays  a  major  part.   The  initial  class,  which  we  terra  professor-intensive,  _s 
characterized  by  class  sessions  in  which  the  professor  takes  the  responsibility 
for  delivering  the  material  or  shaping  the  discussion.   In  student-intensive 
learning,  to  the  contrary,  the  professor  is  either  not  present  at  the  learning 
site,  or  merely  acts  as  a  resource  for  the  learning  student.   In  technology- 
intensive  learning  the  prime  actor  guiding  the  process  is  some  form  of  technol- 
ogy -  today  usually  a  computer. 

From  a  resource  point  of  view,  a  curriculum  developer  can  then  selectively 
utilize  the  strengths  of  these  three  major  actors.   In  sum,  one  can  choose  among: 
(a)   Professors  -  whose  major  comparative  advantage  is  not  in  sheer 

"information  transfer"  but  rather  in  the  development  of  motivation, 
in  the  leading  of  discussions  of  issues  to  which  there  are  no  "right 
answers"  but  whose  significance  must  be  comprehended,  and  in  the 
guiding  of  the  student's  learning  process  through  intelligent  inter- 
action with  students. 
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(b)  Students  -  wliom  it  has  been  shown  are  capable  of  "active"  Learning 
of  well-understood,  well-documented  material  on  their  own  when 
properly  directed  and  relieved  of  the  need  to  sit  "passiveJy"  in 
class,  and 

(c)  Technology  -  which,  it  is  believed,  can  take  the  place  of  previous 
professor  or  student-intensive  methods  where  patient  "answer  checking", 
exploration  of  a  simulated  environment,  or  simple  tutoring  assistance 
is  necessary.  ' 

II.   Specific  Course  Design 

The  traditional  method  of  teaching  this  course  had  been  24  classes  of  1.5 
hours  each  based  on  both  lecture  and  class  discussion  techniques.   As  a  result 
of  the  above-described  process,  the  new  course  structure  was  developed  and  can 
be  summarized  as  shown  in  Exhibit  1.   This  exhibit  illustrates  the  matching  of 
the  types  of  material  to  be  presented  in  the  target  course  (rows)  with  learning 
methods  (columns)  and  with  major  resources  (groups  of  columns).   It  is  a  sparse 
matrix  since  only  the  most  applicable  learning  methods  (the  methods  used)  are 
noted  for  each  type  of  material.   Following  the  resource  lines  shown  in  Exhibit  1, 
the  new  syllabus  was  divided  into  two  basic  types  of  pedagogy.   These  were  a 
classroom-oriented  set  of  lessons  (professor-intensive)  and  self-study  set 
(student  and  technology  intensive)  as  shown  at  the  top  of  exhibit  1  and  described 
below. 

(1)   A  classroom  oriented  set  of  sessions:   To  take  best  advantage  of  the 
professorial  resource,  eight  classroom  sessions  remained  "required".   These 
sessions  (four  at  the  start  of  the  course,  two  in  the  middle  and  two  at  the  end) 
were  timed  and  designed  to  make  maximum  use  of  professorial  skills.   The  start- 
of-the-course  sessions  were  used  to  motivate  the  students  with  regard  to  the 
material,  to  provide  examples  from  the  instructors'  backround  of  the  material's 
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importance  to  managers,  to  provide  an  "overview"  of  the  course,  and  to  develop 
the  decision-making  setting  in  which  the  material  is  utilized  by  managers. 
Basically,  the  intial  sessions  were  used  to  provide  the  students  with  a  "feel", 
from  a  managerial  viewpoint,  of  the  material  they  were  about  to  master.   The 
later  classroom  sessions  were  utilized  to  discuss  case  situations,  providing 
further  perspective  on  basic  material  previously  learned,  as  well  as  linking 
the  material  to  other  areas  of  management  knowledge. 

All  of  these  classroom  sessions  were  designed  around  the  instructors'  com- 
parative advantage  over  other  learning  media.   In  particular,  instructors  -  as 
previously  noted  -  have  advantages  in  terms  of  motivation  of  material;  providing 
students  with  insights  into  "issues"  in  the  course;  exposing  "grey  areas"  in  the 
material;  and,  finally,  in  making  managerial  case  situations  "live"  -  therefore 
further  explicating  and  providing  depth  to  the  material.   Material  sets  1,  2,  5 
and  6  from  Exhibit  1  were  therefore  taught  in  this  manner, 

(2)   Self-study  oriented  sessions:   The  remaining  two-thirds  of  the 
scheduled  course  time  was  devoted  to  a  self-study  mode.   Material  sets  3  and  4 
were  included  here.   As  noted  below,  all  of  the  student-intensive  and  one  form 
of  technology-intensive  learning  methods  were  utilized  in  this  section  of  the 
course . 

It  is  relevant  to  note  that  self-study  was  utilized  -  not  self-paced  self- 
study.   The  students  were  paced  by  the  course  examinations.   It  is  felt  that 
in  their  coming  professional  world,  the  students  will  be  paced  by  externally  im- 
posed deadlines  which  they  will  have  to  meet  by  working  on  their  own.   There  is 
good  reason  to  provide  them  with  experience  in  doing  exactly  this  in  the  academic 
setting. 

For  these  sixteen  self-study  sessions,  students  were  expected  to  study  the 
basic  course  material  on  their  own.   This  self-study  was  assisted,  however,  by 
the  following: 
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(a)  Nine  written  guides  to  the  instructional  modules:   (averaging 
approximately  ten  pages  each)  which  provided  information  as  to  the  (1)  part- 
icular concepts  to  be  learned,  (2)  reading  to  be  done  (both  required  and  optional), 
of  particular  material  in  the  texts,  (3)  the  instructor's  comments  concerning  the 
material  (which  ranged  from  such  things  as  putting  stress  on  particularly  impor- 
tant aspects  of  the  reading  to  further  explanation),  and  (4)  homework  to  be  handed 
in  and/or  other  exercises  to  be  performed,  such  as  on  the  computer. 

(b)  Optional  attendance  at  "Question  Sessions":   Replacing  the  16  class 
sessions  which  would  ordinarily  have  been  held  were  12  "question  sessions"  at 
which  the  instructor  was  present  solely  to  answer  student  questions  on  the  assign- 
ments and  to  respond  to  any  other  questions  concerning  management  in  general  which 
the  students  wished  to  discuss.   Attendance  at  these  sessions  averaged  25%  of  each 
class  (approximately  10  students  per  session) .   Some  two-thirds  of  each  session 
turned  out  to  be  concerned  with  questions  on  the  material.   The  remaining  third 
varied  widely  over  general  management  subjects  -  reflecting  particular  student 
interests. 

(c)  Homework:  This  was  marked  by  the  instructor  and  by  teaching  assistants. 
The  teaching  assistants  also  answered  student  questions  during  their  office  hours, 
TA's  are  traditionally  used  in  this  course. 

(d)  Feedback  examinations:   These  were  brief,  informal,  marked  but  non- 
graded  "quizzes"  which  students  could  take  after  completing  modules  2,  A,  6  and  8 
to  allow  them  to  gauge  their  progress  in  the  course.   The  exams  were  immediately 
marked  and  could  be  discussed  with  the  teaching  assistant. 

(e)  An  interactive  computer  program  -  CLOSE:  This  on-line  computer  program 
allowed  the  student  to  review,  whenever  he  wished  to  do  so,  the  complete  material 
of  the  first  half  of  the  course.   Financial  Accounting  (the  major  material  of 

the  first  half)  breaks  down  into  a  simple  posting-adjusting-closing  cycle  of  events, 
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Using  computer  graphic  capability  on  a  CRT  screen,  students  were  able  to 
test  their  ability  to  perform  financial  accounting  entries  throughout  this 
entire  cycle.   The  interactive  program  assisted  them  by  performing  arithmetic, 
displaying  for  them  the  up-to-date  status  of  the  "books",  and  by  tutoring 
whenever  a  mistake  was  made.   (In  this  last  "tutor"  mode,  the  program  indicates 
each  entry  which  is  made  in  error  -  pointing  to  the  exact  part  of  the  entry  in 
error.   In  addition,  it  provides  "hints"  to  the  student  to  enable  him  to  derive 
the  correct  answer,  but  only  if  the  student  asks  for  "help".   Many  other  similar 
features  of  the  program  provide  it  with  an  ability  to  reasonably  replicate  the 
actions  of  a  human  tutor.) 

The  most  important  point  about  the  program,  however,  is  that  it  enables  a 
student  to  thoroughly  test  his  ability  to  actually  perform  all  of  the  accounting 
functions  which  he  has  learned  in  one  relatively  short  session  at  the  computer 
console.   In  this  way  the  student  can  check  his  detail  and  overall  understanding 
of  the  material  in  a  short  session  with  the  computer.   The  student  can  test  and 
retest  his  knowledge  by  himself,  at  the  time  of  his  own  choosing,  with  or  without 
tutoring  assistance. 

The  material  included  in  the  original  version  of  CLOSE  was  simple  for  the 
best  students  -  but  of  greater  challenge  to  the  less  well  prepared  students  to 
whom  it  is  most  particularly  addressed.   A  version  with  advanced  subject  materia] 
has  been  prepared. 
Process  objectives  for  the  experiment 

The  experimental  course  was  designed  with  five  "process"  objectives  clearly 
in  mind.   These  were: 

(1)   To  improve  academic  productivity  by  making  optimum  use  of  the  strengths 

of  all  of  the  resources  available  to  the  learning  (not  teaching)  process, 
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(2)  To  increase  student  satisfaction  with  the  learning  process  by 
allowing  pupils  greater  flexibility  in  the  use  of  their  time  and 
greater  initiative  in  the  learning  process  through  the  above 
methodology. 

(3)  To  increase  the  quality  of  the  learning  process  by  providing,  for 
those  students  who  desired  it  in  the  particular  subject,  more  mean- 
ingful access  to  the  professor.   (It  is  felt  that  not  al]  students 
want,  or  can  handle,  great  depth  in  all  courses  -  but  greater  depth 
and  breadth  should  be  available  to  those  who  desire  it  in  any  part- 
icular course.) 

(4)  To  allow  (and  encourage)  the  students  to  feel  a  greater  responsib- 
ility for  their  ovm  education  by  putting  the  emphasis,  through  "self- 
study"  and  optional  learning  "feedback"  mechanisms  on  "active"  learnings 
on  the  part  of  the  student  rather  than  passive  classroom-based  absorp- 
tion of  material. 

(5)  To  thoroughly  test  the  validity  of  the  innovative  course  by  both  quan- 
titative and  qualitative  means.  Measures  of  success  were  therefore 
designed  and  control  sections  run  so  that  sound  experimental  results 
could  be  derived. 

III.   Results  of  experimentation  with  the  program 
The  course  was  run  for  the  first  time  in  the  Fall  term  of  1971.   There  were 
seven  sections  of  the  course.   Four  sections  were  taught  in  the  "regular"  fashion  - 
a  combination  of  lecture  and  class  discussion.   Three  of  the  sections  utilized  the 
innovative  program  (hereinafter  these  sections  are  referred  to  as  self-study  or 
"SS",  with  regular  sections  sometimes  referred  to  as  "REG"). 

Students  selected  particular  sections  in  accordance  with  their  schedules. 
Notification  of  which  sections  would  be  self-study  was  not  given  ahead  of  time. 
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In  addition,  every  attempt  was  made  to  play  down  the  "experimental"  nature 
of  the  course.   Although  there  is  undoubtedly  some  Hawthorne  effect  in  every 
innovation,  it  is  felt  that  there  was  as  little  as  possible  involved  in  the 
experimental  results. 

In  order  to  check  the  value  of  the  program  in  an  objective  quantitative 
manner,  the  three  self-study  sections  and  the  four  regular  sections  were  given 
the  same  mid-term  and  final  examinations.   In  addition,  a  questionnaire  was  dis- 
tributed to  all  students  in  both  the  self-study  and  regular  sections  at  the  end 
of  the  course  to  determine  their  subjective  feelings  about  the  course. 

Results,  both  in  terms  of  relative  performance  of  SS  versus  REG  sections 
on  the  examinations  and  in  terms  of  student  "feelings"  about  the  course  as  noted 
on  the  70-question  questionnaire,  suggest  that  the  objectives-material-methods- 
resource  matching  approach  to  the  course  was  a  definite  success.   The  results 
were  as  follows: 

Objective  evidence  from  examination  results:   All  students  in  the  cours>e, 
both  REG  and  SS,  took  the  same  two  examinations  (midterm  and  final)  at  the  same 
time.   The  results  were  as  shown  in  Figure  1. 

The  objective  evidence  from  the  examinations,  both  midterm  and  final,  is 
clear.   The  self-study  sections  outperformed  the  regular  sections.   There  is  no 
reason  to  suspect  that  the  two  groups  were  not  homogeneous  in  mental  ability  and 
background  at  the  start  of  the  course.   Teaching  ability  undoubtedly  makes  some 
unmeasurable  differences,  but  the  average  number  of  years  of  teaching  experience 
in  this  course  of  the  instructors  for  both  SS  and  REG  sections  was  approximately 
the  same.* 


*Still,  one  suspects  that  the  teaching  factor  may  have  played  a  definite 
role  since  significant  differences  in  results  caused  by  learning  method  alone 
are  almost  impossible  to  find  in  the  literature.   See  (15), (16). 
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Mid-Term 


Regular  Sections 


Self-Studv  Sections 


Undergraduate 


Mean 
Std.  Dev. 

N 


42.27 
9.62 
(67) 


Mean 
Std.  Dev 

N 


52.02 
11.28 

(23) 


Graduate 


Mean 
Std.  Dev, 

N 


51.54 
8.84 
(60) 


Mean 
Std.  Dev. 

N 


52.09 

10.56 

(82) 


TOTAL 


Mean 
Std.  Dev, 

N 


46.65 
10.22 
(127) 


Mean 
Std.  Dev, 

N 


52.08 
10.67 
(105) 


Final 


Regular  Sections 

Self-Studv 

Sections 

Undergraduate 

Mean 

Std.  Dev. 

N 

87.66 

19.48 

(57) 

Mean 

Std.  Dev. 

N 

101.55 

21.36 

(21) 

Graduate 

Mean 
Std.  Dev. 

N 

102.17 
20.1 
(57) 

Mean 

Std.  Dev. 

N 

105.10 

25.45 

(82) 

TOTAL 

Mean 
Std.  Dev. 

N 

94.91 
21.02 
(114) 

Mean 

Std.  Dev. 

N 

104.37 
24.61 
(103) 

Figure  1 
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The  differences  for  the  TOTAL  (both  undergraduate  and  graduate)  scores 
of  SS  versus  regular  (52.08  vs.  46.65  on  the  midterm  and  104.37  vs.  94.91  on 
the  final)  is  highly  significant  at  the  .01  level  on  a  t-test.   On  both  exam- 
inations, SS  undergraduates  and  graduates  outperformed  their  REG  counterparts. 
But  the  difference  is  most  striking  with  regard  to  undergraduates.   However, 
when  a  group  of  graduates  who  took  the  course  in  a  "regular"  section  as  an 
elective  (and  who  therefore  might  be  expected  to  be  more  motivated  toward  tne 
material)  is  removed  from  the  data,  the  examination  results  show  a  significant 
difference  in  the  graduate  ranks  in  favor  of  self-study,  also'. 

Subjective  evidence  from  questionnaires;   In  the  free-form  section  of  the 

course  questionnaire,  self-study  students  provided  some  logic  behind  the  above 

results.   Some  of  the  ideas  frequently  expressed  were: 

"I  felt  the  responsibility  for  learning  the  material  had  been  trans- 
ferred, under  self-study,  from  the  professor  to  me." 

"Self-study  is  a  lot  more  productive.  I  didn't  have  to  waste  time 
in  class  on  things  I  already  understood .. .or  spend  time  in  class  when  I 
was  too  far  behind  for  the  class  to  do  me  any  good." 

"The  computer  program  enabled  me  to  check  on  my  own  learning  of  the 
entire  first  half  of  the  course  when  I  was  ready  to  do  so.   That's  great. 
You  ought  to  have  a  program  for  the  second  half,  too,  if  you  can  design  one. 
It  helps  to  be  able  to  review  all  the  material  -  for  integrated  understand- 
ing -  in  2-3  hours  on  the  computer." 

The  instructors  liked  it,  too.   One  self-study  instructor  summarized  his 

views  by  saying: 

"I've  never  enjoyed  teaching  so  much.   I  taught  only  those  things  which 
require  a  professor.   Self-study  removed  the  'transmission'  of  a  lot  of  the 
dreary  details  which  can  be  learned  from  a  book.   The  question  sessions,  above 
all,  were  challenging  and  fun.   Only  those  students  who  were  motivated  to  come 
showed  up  -  and  our  discussions  were  always  lively". 

"We  got  to  discuss,  in  addition  to  the  strict  course  material,  things  like 
management  in  general,  and  philosophies  of  life  -  both  mine  and  theirs.   These 
were  things  one  would  hesitate  to  tie  up  a  class  of  40  on  -  but  with  10-12  inter- 
ested students,  these  discussions  came  naturally." 
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"The  variety  of  possible  questions,  some  of  which  I  could  not  answer 
(students  could  ask  anything)  somehow  made  me  appear  more  human,  I  think. . . 
a  bit  less  the  'authority'  and  a  bit  more  an  'assistor  in  the  learning  process.' 
This  is  the  only  way  to  teach.   The  students  who  want  personal  contact  get  more 
attention  than  they  would  otherwise  get  (10  people  in  question  sessions  vs.  40). 
Those  who  are  satisfied  merely  with  picking  up  the  basic  knowledge  alone  in 
this  particular  course  are  left  alone  to  do  so  most  easily  and  efficiently." 

More  quantitative  assessments  of  these  "feelings"  are  available  from  the 
course  questionnaire  data.   Student  answers  to  eight  of  the  questions  tell  much 
of  the  story.* 

As  can  be  seen  from  the  data  in  Figure  2,  the  students  in  the  self-study 
sections  heavily  favored  self-study  as  a  method  of  learning  the  material 
(question  1).   Interestingly,  the  students  in  the  regular  sections  -  many  of 
whom  had  seen  the  SS  materials  or  talked  with  their  classmates  about  it  -  also 
favored  self-study.   Some  8%  of  SS  students  did  not  favor  SS  -  which  says,  at 
least,  that  learning  styles  differ. 

This  particular  course  has  always  been  seen  as  valuable  by  students.   Tt 
maintained  this  ranking  with  students  this  term,  but  was  especially  favored  by 
self-study  students  (Figure  2,  question  2) . 

Since  the  course  is  aimed  primarily  at  transmitting  "techniques"  and 
"principles",  it  is  rewarding  to  note  that  self-study  and  regular  students  per- 
ceive the  course  as  performing  this  task  (questions  3,  4).   Self-study  students 
perceived  a  greater  change  in  their  knowledge  of  management  principles  and  tech- 
niques.  Interestingly,  however,  although  it  might  be  expected  that  the  study- 
by-yourself  mode  would  lead  only  to  knowledge  of  "facts",  self-study  students 
(with  a  minimum  of  classes)  also  clearly  felt  that  they  were  more  stimulated  to 
think  about  the  "issues"  than  did  students  in  the  regular  sections  (question  5)1 
It  is  hypothesized  that  the  available  classroom  time  is  more  intensively  utilised 
to  focus  on  important  issues  by  both  professor  and  students. 

*Seven  point  scales  were  used.   For  clearer  exposition,  and  to  allow  Chi 
Square  analysis,  the  points  were  grouped  to  a  three  point  scale. 
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"For  learning  the  material 
in  this  course,  self-study 
is  a  preferable  mode." 


Strongly! 
Agree  |  Uncertain 

Strongly 
Disagree 

ss 

71 

21 

8 

REG 

41 

42 

17 

Chi  Square 
Significance  Level 

.005 


"This  course  will  be  more 
useful  tu  me  in  the  future- 
compared  with  other  cour- 
ses ." 


Strongly 
Agree 

Uncertain 

Strongly 
Disagree 

SS 

50 

40 

10 

REG 

35 

49 

16 

Chi  Square 
Significance  Level 

NS 


"During  the  course  my 
knowledge  of  management 
techniques  changed  con- 
siderably." 


Strongly 
Agree 

Uncertain 

Strongly 
Disagree 

SS 

70 

28 

2 

REG 

49 

46 

5 

Chi  Square 
Significance  Level 

NS 


"During  the  course  my 
knowledge  of  management 
principles  changed  con- 
siderably . " 


Strongly 
Agree 

Uncertain 

Strongly! 
Disagree! 

SS 

59 

55 

i 

6    ! 

REG 

49 

45 

6   1 

Chi  Square 
Significance  Level 

NS 


"In  this  course,  the 
professor  stimulates 
students  to  think  about 
the  issues." 


Strongly 
Agree 

Uncertain 

Strongly; 
Disagree! 

SS 

46 

51 

1 
3    j 

REG 

19 

56 

25    i 

Chi  Square 
Significance  Level 

.005 


"Professor  has  a  clear 
plan  for  the  semester's 
work." 


No 

Uncertain 

Yes 

SS 

4 

16 

80 

REG 

25 

42 

35 

Chi  Square 
Significance  Level 

.005 


"Class  sessions  are  al- 
ways interesting." 


No 

Uncertain 

Yes 

SS 

25 

50 

25    1 

REG 

49 

49 

^    1 

Chi  Square 
Significance  Level 

.005 


"Professor  motivates 
student  interest  in  the 
material . " 


No 

Uncertain 

Yes 

SS 

7 

44 

48 

REG 

46 

49 

5 

Figure  2 

SIGNIFICANT  QUESTIONNAIRE  RESULTS 
(A  pertinent  set  of  9  of  the  70  questions) 


Chi  Square 
Significance  Level 
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"Professor  is  available 
for  outside  assistance." 


; 

No 

Uncertain 

1 

'  Yes 

ss 

13 

1 

i     34 

'       53 

REG 

28 

1 

!   13 
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Finally,  it  was  clear  that  the  self-study  mode  forces  the  professor  to 
invest  more  heavily  in  thinking  through  the  course,  planning  it  carefully, 
and  having  a  clear  idea  as  to  the  goals  and  subgoals  of  the  course.   This 
effect  was  evident  to  the  students  as  shown  by  responses  to  questions  6-8  in 
Figure  2. 

The  course  design  and  the  materials  for  implementation  purposes  required 
approximately  four  man-months  of  effort.   As  noted  below,  the  resulting  increase 
in  academic  production  can  quickly  ammortize  this  investment.-^ 

Relationship  to  academic  productivity 

The  course  allows  a  50%  increase  in  academic  productivity.   One  instructor 
can  now  handle  three  sections  instead  of  two.   This  potential  gain  will  be  im- 
plemented in  the  Fall  1972  term  -  the  next  time  the  course  is  taught. 

This  statement  concerning  productivity  is  based  on  the  conclusion  that  the 
average  number  of  students  in  each  question  session  can  be  increased  from  10  to 
15  with  little  effect.   Thus  one  instructor  can  "lecture"  three  sections  and 
combine  them  into  two  for  "question  session"  purposes  -  providing  an  exactly 
equivalent  teaching  hour  load  for  the  instructor  as  in  the  past  with  a  507  in- 
crease in  sections  taught.   Briefly: 

Old  System  (2  sections)  New  System  (3  sections) 

Sections  =   2  Lecture  sections       =  3 

Class  sessions       =  24  Lecture  class  sessions  =  8 

Hours  per  session    =   1.5  Hours  per  session      =  1.5 

(2  X  24  X  1.5)       =  72  class  hours  (3  x  8  x  1.5)  3^  hours 


*in  addition,  some  course  redesign  takes  place  each  year.   Thcrerore,  the 
four  months  of  curriculum  development  effort  is  not  all  "extra"  fffort. 
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Question  sections         =   2 

(At  15  students  per       =  12 
session  average) 

Question  sessions  =  12 

Hours  per  session  =  1.5 

(2  X  12  X  1 .5)  =  36  hours 

TOTAL  LOAD  (36  +  36)  72  hours 

During  this  term  there  was  evidence  that  the  increase  in  students  in  the 
question  sessions  from  an  average  of  10  to  15  was  highly  feasible.   A  very  few 
sessions,  did,  in  fact,  have  15  students.   These  sessions  were  felt  in  no  way 
to  be  inferior. 

Relationship  to  educational  quality 

It  is  felt  that  this  course  has  increased  the  quality  of  education  in 
several  ways.   First,  students  in  the  question  sessions  had  a  chance  to  follow 
their  agenda.   They  could  inquire  about  what  interested  them  -  whether  or  not 
it  was  part  of  the  strict  curriculum.   And  the  students  took  advantage  of  it. 
Questionnaire  data  reflect  their  belief  that  the  course  was  better  planned  and 
better  orchestrated  for  them.   Second,  and  perhaps  most  important,  the  individual 
students  who  most  wanted  exposure  to  the  instructor  in  this  particular  course  re- 
ceived it  in  the  self-study  group.   Twenty-five  per  cent  of  the  class,  on  the 
average,  attended  the  "question"  sessions  and  received  attention  as  if  the  40  man 
class  were  a  10  man  seminar!   In  the  questionnaire,  "access  to  the  instructor" 
was  seen  to  be  significantly  greater  (.001  level)  in  the  self-study  sections  than 
in  the  regular  sections  (Exhibit  2,  //9)  .   Some  students  noted  that  the  ccjmputer 
program  allowed  them  to  get  an  integrated  understanding  of  that  section  of  the 
material  which  they  felt  was  not  available  in  previous  courses  tJiey  had  taken. 
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This  particular  course  deals  with  basic  material.   Its  decreased  need 
for  faculty  time  will  also  provide  faculty  availability  to  teach  more  pro- 
fessor-dependent material  in  later  courses  in  smaller  sections.   Alterna- 
tively, greater  faculty  time  will  be  available  for  curriculum  development 
to  introduce  new  concepts  and  material. 

SUMMARY 

The  educational  literature  has  recorded  many  advances  in  technique  over 
the  past  few  years.   Some  have  stressed  technology.   Others  have  stressed 
more  av^;areness  of  human  potential  and  psychology  in  the  learning  process. 
Unfortunately,  most  of  these  advances  have  been  applied  one  at  a  time. 

This  paper  suggests  that  the  strengths  of  each  of  the  three  major  avail- 
able resources  in  the  learning  process  must  be  clearly  understood  and  the  use 
of  these  three  resources  carefully  orchestrated  in  each  course.   The  exact  use 
of  each  resource  will  depend  on  previously  defined  course  objectives,  the  trans- 
formation of  these  objectives  into  relevant  classes  of  materia],  and  the  methods 
which  are  chosen  to  best  transmit  each  type  of  material.   We  know  a  great  deal 
about  this  process,  but  it  is  far  from  a  science  at  present. 

An  illustration  of  the  recommended  process  as  performed  for  a  particular 
course  has  been  given.   It  suggested  that  careful  use  of  the  process,  resulting 
in  an  integrated  and  selective  use  of  the  available  resources,  can  result  in 
greater  academic  productivity,  while  at  the  same  time  increasing  both  perceived 
course  quality  and  student  satisfaction  with  the  course. 

Of  some  importance  is  the  fact  that  this  educational  change  is  implementable 
in  today's  universities.   It  is  not  a  radical  departure,  yet  it  p,oes  far  toward 
meeting  some  of  the  currently  felt  needs  in  education  as  noted  in  the  first  sev- 
eral paragraphs  of  this  paper.''-' 


*See  Schein  (17)  for  a  discussion  of  the  importance  of  this  last  factor 
the  need  to  fit  educational  innovation  into  the  present  system. 
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